The present invention relates to an image processing apparatus and, more particularly, to an image processing apparatus for generating high-quality image data having all color information for each pixel from original image data sensed by a single-CCD color electronic camera device or the like.
Generally in an image sensor such as a CCD (solid-state image sensor) for performing photoelectric conversion for many pixels arrayed in a matrix on a two-dimensional plane, if any pixel is defective, this defective pixel cannot attain a correct output, i.e., pixel value. In the use of such an image sensor, the pixel value of the defective pixel must be compensated using the pixel values of pixels around the defective pixel.
In general, in an image sensing device using a single multi-color filtered image sensor, image data having all color information (RGB or a luminance value and color difference signals) for each pixel is generated by interpolation processing from original image data of an almost checkered pattern having only a pixel value of one original color (e.g., any one of R, G, and B) for each pixel. If original image data obtained by the image sensor contains the pixel value of a defective pixel, the influence of the defective pixel diffuses to neighboring pixels, degrading the image quality.
Conventionally, to compensate for the pixel value of such a defective pixel, it is replaced using the pixel value of a pixel of the same color one or several pixels before the defective pixel, or the average of pixels of the same color before and after the defective pixel. Further, the pixel value of the defective pixel is adaptively replaced based on various conditions using a combination of the pixel values of two or more pixels before and after the defective pixel (see, e.g., Japanese Patent No. 2,636,287 and U.S. Pat. No. 5,805,216).
These pixel value replacement processes can make the defect of original image data obtained by an image sensor less conspicuous. However, a single multi-color filtered image sensor must execute the above-described interpolation processing after replacement processing. Interpolation processing after replacement diffuses the pixel value of the defective pixel replaced using neighboring pixel values to neighboring pixels, and high-quality image data cannot be obtained.
According to a method of replacing the pixel value of a defective pixel using the pixel value of a pixel of the same color positioned immediately before the defective pixel, when a defective pixel G43 exists near the boundary between a bright region 101 and a dark region 102, as shown in FIG. 21A, the pixel value of the pixel G43 is replaced by that of the pixel G23 of the same color positioned immediately before the pixel G43. As shown in FIG. 21B, the pixel value of the pixel G43 which should originally represent a bright pixel is replaced by a pixel value which represents a dark pixel. The pixel value of the replaced pixel G43 is used to generate a pixel value of G color at a neighboring interpolation point in subsequent interpolation processing. Thus, false information diffuses in accordance with the size of the interpolation region.
More specifically, if a defective pixel exists in an interpolation region used to obtain each color information at an interpolation point, predetermined color information contains the pixel value of the defective pixel, so that the pixel value of the defective pixel diffuses to all interpolation points using the interpolation region containing the defective pixel. In an interpolation region of 3×3 pixels centered on an interpolation point, a dark region of 3×3 pixels centered on the pixel G43 appears in the bright region 101, as shown in FIG. 21C. High-order interpolation processing using a wider interpolation region in order to ensure a high image quality further widens the influence.
This phenomenon occurs due to not only pixel value replacement processing for compensating for a defective pixel of an image sensor, but also interpolation processing performed after pixel value replacement processing.